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Abstract 

The form invariance of pseudoadditivity is shown to determine the structure of nonextensive 
entropies. Nonextensive entropy is defined as the appropriate expectation value of nonextensive 
information content, similar to the definition of Shannon entropy. Information content in a nonex- 
tensive system is obtained uniquely from generalized axioms by replacing the usual additivity with 
pseudoadditivity. The satisfaction of the form invariance of the pseudoadditivity of nonextensive 
entropy and its information content is found to require the normalization of nonextensive entropies. 
The proposed principle requires the same normalization as that derived in [A.K. Rajagopal and S. 
Abe, Phys. Rev. Lett. 83, 1711 (1999)], but is simpler and establishes a basis for the systematic 
definition of various entropies in nonextensive systems. 
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I. INTRODUCTION 



Since the first proposal of nonextensive entropy by Tsallis JT|, there have been many 
successful studies and applications analyzing physical systems such as long-range interac- 
tions, long-time memories, and multi-fractal structures in the nonextensively generalized 
Boltzmann-Gibbs statistical mechanics || . In the rapid progress in this field, some modifica- 
tions have been made to mathematical formulations in the generalized statistical mechanics 
in order to maintain the physical consistency. One of the most important modifications was 
the introduction of an appropriate definition of the generalized expectation value. This mod- 
ification has already appeared in the literature M, but has been applied as a candidate for 
satisfying the physical requirements in a given situation without a systematic framework. 
The necessity to apply such modifications invites the establishment of guiding principles 
that will provide a clear basis for proposed generalizations of Boltzmann-Gibbs statistical 
mechanics. In a recent paper ||, Rajagopal and Abe presented a principle for determin- 
ing the structure of nonextensive entropies. Their principle was the form invariance of the 
Kullback-Leibler entropy when generalized to nonextensive situations. 

In the present paper, a much simpler principle for determining the structure of nonexten- 
sive entropies is presented. The original Tsallis entropy is determined for the given appro- 
priate axioms |], [7], [8|. In contrast to these axiomatic approaches, we define nonextensive 
entropy in another way; in terms of the appropriate expectation value of nonextensive infor- 
mation content similar to the definition for Shannon entropy |J . This definition has already 
been applied to the generalization of the Shannon source coding theorem using the normal- 
ized q- expectation value of nonextensive information content ||10|| . Nonextensive information 
content I q (p) is defined by I q (p) = — ln q p in flO| as an intuitively natural generalization 
of the standard information content I\ (p) = — Inp (referred to as self-information or self- 
entropy in Shannon information theory @)), where ln,j x is a q- logarithm function defined by 
\n q x = (x l ~ q — 1)/(1 — q)- However, in flO |, the form invariance presented in this paper was 



not mentioned. We introduce the axioms for nonextensive information content I q (p) as a 
slight generalization of that for the standard information content, and obtain I q (p) uniquely 
from the generalized axioms. The requirement of form invariance of pseudoadditivity when 
we define nonextensive entropy S g (p) as the appropriate expectation value of I q (p), 

S q (p)=E qiP [I q ( Pi )], (1) 



leads to the determination of the structure of the nonextensive entropy, where E q>p [ • ] is 
the expectation value satisfying the following form invariance of the pseudoadditivity: 
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where A; is a positive constant, ip (q) is any function of the nonextensivity parameter q and 
satisfies the conditions (0) given below. 

Note that information content means the amount of information provided by a result of 
an observation in a physical sense. The standard information content I\ (p) = — Inp has 



been called surprise by Watanabe 11 , and unexpectedness by Barlow ||12||. 



II. NONEXTENSIVE SELF-INFORMATION 

The axioms of standard information content I\ : [0, 1] — * R + satisfying I\ (1) = are 
given as follows |§: 

[SI] Ii is differentiable with respect to any p e (0, 1), 

[S2] h (P1P2) = h (Pi) + h (pa) for any p u p 2 E [0, 1]. 

Axiom [S2] means that the information content for two stochastically independent events 
is given by the sum of the two sets of information. 

For the above axioms, I\ (p) is determined uniquely by 

1\ (p) = —A; lnp (4) 

where k is a positive constant 0. 

The above axioms are generalized in nonextensive situations as follows. 
Nonextensive information content I q : [0, 1] — > R + for any fixed q G R + , satisfying 

lim Op) = Ii Op) = —A; lnp, (5) 

q— *1 

should have the following properties. 
[Tl] I q is differentiable with respect to any p G (0, 1) and q G R + , 



[T2] I q (p) is convex with respect to p G [0, 1] for any fixed q G R + , 



[T3] there exists a function </? : i? — > i? such that 

Iq(PlP2) Iq(px) , ^(P2) 
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for any pi,p2 £ [0, 1], where (g) is different iable with respect to any q G 
lim^M^O, lining) = y?(l) = 0, and y> (g) ^ (q ^ 1) . 



(6) 



(7) 



Equation @ is called pseudoadditivity in many studies ||, as a special form of compos- 
ability 0, 0. 

Note that in these generalized axioms, [T2] is needed to maintain nonnegativity of the 



Kullback-Leibler entropy for any q G R + when generalized to nonextensive situations [15, 16 



In general, Kullback-Leibler entropy K yp A p B J is defined by the appropriate expectation 
value of the difference between two information contents HI. 



I (pf) - / (pf 



(8) 



Therefore the nonnegativity of the Kullback-Leibler entropy leads to Gibbs inequality [T7], 
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When E p is a normalized expectation value (i.e., E p [1] = 1) and p B = 
right-hand side E p A I (pfj of the above Gibbs inequality is equal to the maximum entropy. 
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(10) 



This inequality coincides with the maximality condition which is one of the Shannon- 
Khinchin axioms [19|, |2(|, that is, the simplest form of the maximum entropy principle 
without constraints. Therefore the satisfaction of the nonnegativity of the Kullback-Leibler 
entropy for any q G R + is needed when generalized to nonextensive systems. In order to 
satisfy the requirement of the nonnegativity, the information content I q (p) should be convex 
with respect to p G [0,1] for any fixed q G R + and its corresponding appropriate expectation 
value should be applied. Such examples of expectation value are q-expectation value and nor- 
malized q-expectation value P, ffl. Under these two conditions (convex information content 
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and appropriate expectation value), the nonnegativity of the nonextensive Kullback-Leibler 
can be proved. The proof is given in appendix A. 

Using the axioms [T1]~[T3], we determine I q (p) uniquely in the following procedures. 
Using (|(J), for any 1 + A G (0, 1), we have 

I q (p(l + A)) _ I q (p + Ap) _I q (p) 7,(1 + A) I q (p) / g (l + A) 

h h h h * " ' h h ' ^ ' 

th rv rv rv ru rv 

This can be rewritten as 

I q (p + Ap)-I q (p) 1 I q (l + A) k + V (q)I q (p) 



Ap k A p 

Taking the limit A — > on both sides of equation (|12"D, we obtain 

dl q (p) _ k + cp (q) I q (p) 



(12) 



(13) 



dp k p 

where (3 = ^lim Jg ^ A ^ and the first axiom [Tl] is applied. The differential equation is 
given by 

, , 1 / x dy = j ■ -dx (14) 
k + (p{q)y k x 

where x = p and y = I q (p). This can be solved analytically; the rigorous solution is 

•p(i) . •p(i) 
(CxP) k -1 (CpP) k -1 
y = h-± J —- , that is, I q (p) = k-± J —- (15) 

where C is a constant. Then, from the initial condition lim/„(p) = I\{jp) = — klnp, we 

q-*l 

have that C = 1 and (3 = —k, where conditions (|7|) are applied. Thus, the nonextensive 
information content I q (p) is derived as I q (p) = k ■ p ^F^ - 

Moreover, by [T2], the second differential of I q (p) with respect to p should be nonnegative 
for any fixed q G R + . Thus, we can derive a constraint <p (q) + 1 > for any q G R + . 

Summarizing these results, the nonextensive information content I q (p) obtained from the 
axioms [T1]~[T3] is 

n -v(?) _ i 

h{p)=k --^r (16) 

where k is a positive constant and 

¥ (q) + 1 > for any q G R + . (17) 
For example, tp (q) — q — 1 implies I q (p) = — khi q p. 
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Note that there have been already remarks on the alternative candidates for nonextensive 
information content to define the original Tsallis entropy They are 

J« (p) = -kki q p and lj® (p) = kln q p~ l , (18) 

and they only coincide for q — 1. 

w 

1 — Pi w w 

s? rg ip) = k- = £p?/« (p^ = E^ 2) (pO (i9) 

y 1 i=l i=l 

(p) and 1^ (p) correspond to ip^ (q) = q — 1 and <£>( 2 ) (g) = 1 — g as y? (g) in (|Tt3[), 
respectively. However, the latter case (p^ (g) = 1 — g does not satisfy the identity ( |TTD for 
any g G that is, J^ 2 -* (p) does not possess the property of convexity [T2]. Therefore, 
7^ 2 ) (p) cannot be information content. Even if l^ 2 ) (p) is applied as information content, 
then the nonnegativity property of the nonextensive Kullback-Leibler entropy is not held 
for any q G R + for lack of the convexity of 1^ (p), as stated above. The convexity of 
information content is applied to Jensen's inequality in order to prove the nonnegativity of 
the nonextensive Kullback-Leibler entropy. See the appendix A for the details. 

In case of tp (g) = g — 1 and k — 1, the pseudoadditivity (j6|) of I q (p) is remarkably 
similar to the relation of the Jackson basic number in g-deformation theory |21], |22|] as 
follows. Let [X) q be the Jackson basic number of a physical quantity X, that is, [X] = 
( y q x — ]}j J {q — 1). Then the Jackson basic number of X + Y satisfies the identity [X + Y\ q = 
[X] + [Y] + (g — 1) [X] [Y] . The surprising similarity to pseudoadditivity (|5|) can be seen 
if we consider a quantity / (x) = p~( x_1 ). Clearly, / (1) = 1. Standard information content 
Ii (p) is expressed as I\ (p) = df (x) /dx\ x=1 ; nonextensive information content is given by 
Iq (p) = D q f (x)\ x=1 = (/ (qx) — f (x))/(qx — x)\ x=l , where D q is the Jackson differential. 
According to g-deformation theory, the property lim 9 ^! I q (p) = Ii (p) originates from the 
convergence lim^^i D q = d/dx. 

III. EFFECTS OF RENORMALIZATION OF NONEXTENSIVE ENTROPIES 

The normalized nonextensive entropies follows naturally from the form invariance between 
entropy and its information content. In this section we assume k = 1 for simplicity. 

Similar to Shannon entropy, nonextensive entropy S q (p) is defined as the expectation 
value of the information content I q (p) obtained in fllCf). For example, the nonextensive 
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entropy S q using the unnormalized expectation value E q , p [ ■ } is given by 



s? rg Cp) = *S g ft (p)] = E *? (r) (20) 

i=l 

where W is the total number of microscopic configurations with probabilities {pi}- In the 
definition of in fl20f) , the q-expectation value |2|, |j] is used. If we let (g) = g — 1, then 
S° T ^ (p) is concretely derived from (|i~6l) and ( p0|) as follows: 

l - E p 9 

^ org (p) = ^fr 1 - ( 21 ) 

This is the original Tsallis entropy |L|. 

Let A and B be two independent systems in the sense that the probabilities pf B of the 
total system A + B factorize into those of A and of B, i.e., 

P?j B = pfpf for any % = 1, • • • , W A and j = 1, • • • , Wb. (22) 

The nonextensive entropy >S° rg (p AB ) of the total system A + B can then be expanded using 
definitions (|20|) and (^) as follows: 

Wa W b Wa W b 

S ™ g (p^) = £gg [J 9 (p^)] = ^ ^ (p^)« / 9 (pjJB) = ^ £ (pfp ? ) V ? (p^p ? ) (23) 

i=l j=i j=l j=i 

Applying the pseudoadditivity @ for information content I q (p), we obtain 

W A W b 

s or g ^ = E E ( pfp *y { J? + Jg ^ + ^ {q) Iq ^ Iq ^ | 

i=i i=i 

= ( E (Pf )1 ^ (P A ) + IE (pt Yj S {P B ) + v(d)S (p A ) S ( P B ) (24) 

where we used 

W A n % 

^° rg {P A ) = E (rf ) h (it ) and ^° rg (p B )=E(pf) ^(Pf)- (25) 
i=i i=i 

Dividing both sides of (El) by 



i=l i=l \i=l / \i=i / 

yields 

v v + w „ - +p(g) ^ v - - ■ (27) 



W A W S/ , W A „ ' W B , , n ' ^ ^> W A A „ W B 

E E fef ) E (pf) 9 E pf E (P?) q Y 

i=l j=l i=l j=l i=l j"= 
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In order to preserve the form of pseudoadditivity between nonextensive entropy S q (p) and 
the corresponding information content I q (p), the nonextensive entropy is modified as 

w 



Sf° T (p) = = ^ (28) 

E Pj Ep] 

3=1 3=1 

which is the expectation value of the information content I q (pi) with respect to the escort 



distribution [24] of p of order q. Then, by substituting (|2q ) into (|27|) , the following equation 



with respect to S^ or (p) is obtained. 

Sf OT (P AB ) = Sf° T (/) + Sf™ (P*) + V (?) (/) ^ nOT {P B ) ■ (29) 

This is the same pseudoadditivity of nonextensive entropy S q (p) as (||). We have thus 
derived from the expectation value of I q (p) the form invariance of the pseudoadditivity 
of nonextensive entropy and its information content. Moreover, the nonextensive entropy 
^nor ^ Qigfingo! D y (pgj) is actually the q-normalized nonextensive entropy [[5], p3[ . Thus, 



according to the principle of the form invariance of pseudoadditivity between nonextensive 
entropy S q (p) and its information content I q (p) , Si 101 (p) should be used as nonextensive 
entropy instead of S q r ^ (p). If E™ 1 denotes the normalized g-expectation value with respect 
to {pi}, defined by 



w 

F OTg r.-, E Pi A 

\ P i A \ = — ~w ( 3 °) 

E Pj E Pj 

3=1 3=1 

where A is a physical quantity, then g-normalized nonextensive entropy Si 1010 (p) is given by 

sf or (p) = K° T (ft)] ■ (31) 

For the most typical case <p(q) = q — I, S^- OT (p) is concretely given by 

w 

l - E p 9 

Sf° T ( P ) = (32) 

(q ~ 1) E P) 

3=1 



This normalized Tsallis entropy (32) is concave only if the nonextensivity parameter q lies 
in (0,1) i|. 

Note that definition ( p0| ) can be easily replaced with a more general unnormalized ex- 
pectation value, leading to almost the same conclusion as that derived in this study. For 
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example, if we use a more general form (|TJj) as information content, then the expectation 
value Ef,p w 6 defined by (Q can be applied to the definition of the generalized original 
Tsallis entropy Sf org . 

w (1 i-E P r" ,+1 

Sf or S (p) = [/, (p.)] = £p*> +1 /, « = . (33) 

Then, along the same procedure as that presented in this section, the following Sf nor can be 
obtained in order to preserve the form invariance of pseudoadditivity between nonextensive 
entropy and its information content. 

oS-org / n \ 1 - E Pi 
5 g-nor / n ^9 W = i=i / 34 n 

E pf )+1 v (?) E vf q)+1 
j=i i=i 



In fact, when (g) = g — 1, the formulas (^) and ( |34|) coincide with and respec- 
tively. 

A dissatisfaction of the form invariance of the pseudoadditivity in the original Tsallis 
entropy can be revealed through the following simple calculation. Here we take k — 1 for 
simplicity. When <p(q) = q — 1, and substituting ( |L6j) into (p0|) , S^ 1 ^ (p) coincides with the 



original Tsallis entropy as shown in (^l|) . The original Tsallis entropy Sq T (p) given by 
(PI) is widely known to satisfy the following pseudoadditivity ||: 

Sf g (p AB ) = (p A ) + Sf* (p B ) +(l-q) (p A ) 5f« (p*) (35) 

However, the pseudoadditivity @ of J g (p) for the same condition (i.e., (p (q) — q — 1) is 
given by 

ipm) = h (pi) + /, (P2) + (g - 1) /, (pi) I q (pa) • (36) 

By comparing ( |3"5] ) and fl3"E|), the coefficient (5 — 1) of the cross term of pseudoadditivity 
(Effi) differs from the (1 — g) in (E3) when [ • ] defined by (BO) is used. This clearly 



reveals that the form of the pseudoadditivity of S q (p) and I q (p) is not invariant in the 
computation of E q , p [ ■ ] . In other words, the form of the pseudoadditivity is not fixed when 
the unnormalized expectation value E g , p ° [ ■ ] is applied to the definition of Tsallis entropy. 

More generally, for the generalized original Tsallis entropy Sq obtained in (p3|) , the 
following pseudoadditivity is held. 

Sf ° rg (p AB ) = Sf-™Z (p A ) + Sf-™Z (p B ) -<p(q). Sf-^ (p A ) Sf-™Z (p B ) (37) 



By comparing (J6|) and (|3~7D , a "y? (<?) versus —tp(q) inconsistency" can be found as similar 
as the above discussion. Note that when q — 1, the form invariance discussed here holds 
because both Eg,p & [ ■ } and £ 9 V 6 [ • ] become a normalized expectation value when q = 1. 

Therefore the unnormalized expectation value such as E q , p [ ■ ] result in an inconsistency 
in the form invariance of the pseudoadditivity for the original Tsallis entropy. 

If we let f (q) = q — 1, then from ( |2UD the following pseudoadditivity holds. 

Sf or ( P AB ) = Sf™ (p A ) + (p B ) + (q-l) (p A ) ( P B ) (38) 

In other words, S^- or (p) given by ( |32l) satisfies the same pseudoadditivity (|38| ) as (|36l). 
Therefore, the form invariance of pseudoadditivity requires the change from the familiar 
identity (^) to the modified one (38). This follows clearly from the above discussion, because 
when [ ■ ] defined by (|20|) is applied to the definition of S q (p), the form invariance of 
the pseudoadditivity is not held as shown above. 

Note that the obtained pseudoadditivity (|38| ) is same as the relation of the Jackson 
basic number: [X + Y] q = [X] q + [Y] q + (q - 1) [X] q [Y] q where [X] q = (q x - l) /{q - 1) 
0. Consider a quantity f (x) = 1 ^J2(pi) x - Clearly, / (1) = 1. Shannon entropy Si(p) 
is expressed as Si (p) — df (x) j dx ; i 
D q f (x) | = (/ (qx) - / (x)) /(ga; - x) 



is expressed as 5*i (p) = df (x) j dx ; normalized Tsallis entropy is given by S^ 01 (p) 



, where D n is the Jackson differential. 



IV. CONCLUSION 

We have established a self-consistent principle for the form invariance of the pseudoaddi- 
tivity of nonextensive entropy and its information content. The present principle is drawn 
from Shannon information theory and leads to the same normalization of the original Tsallis 
entropy as that derived in ||. 

Once a set of an information content and an expectation value is given, various entropies 
such as Kullback-Leibler entropy (relative entropy) and mutual entropy can be formulated 
systematically. In nonextensive systems, an information content ( fL6[) and two expectation 
values fl20D and (|30|) are given in the previous sections. Therefore we can formulate two sets 



of various entropies based on two sets of the information content (|16|) and the expectation 



value (pQD or (p0[), respectively. Please see the concrete formulas in appendix B. 



Note that the alternative selection from the original Tsallis entropy or the normalized 
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Tsallis entropy should be careful in each application. From the mathematical point of 
view, the normalized Tsallis entropy has nice properties such as the form invariance of the 
pseudoadditivity, the unified application of the normalized g-expectation value, the form 
invariance of the statement of the maximum entropy principle and so on. However, from the 
physical point of view, the original Tsallis entropy has many advantages over the normalized 
version. For example, the results derived from the Kolmogorov- Sinai entropy using the 
original Tsallis entropy have the perfect matching with nonlinear dynamical behavior such 
as the sensitivity to the initial conditions in chaos. On the other hand, the normalized Tsallis 



entropy does not have these convenient properties. Please see the references p5| ^6], [27], |28 
for the details. 

The principle discussed here is based on the usual formulation for information in Shan- 
non information theory. Therefore, the ideas presented in this paper are an application of 
information theory to statistical mechanics, similar to the philosophy of Jaynes' work J29) . 
There remain many other applications of Shannon information theory to this interesting 
field. 



Acknowledgments 

The author would like to thank Prof. Yoshinori Uesaka and Prof. Makoto Tsukada for 
their valuable comments and discussions. The author is grateful to Prof. Sumiyoshi Abe 
for reading the first draft and his useful comments. The author is very much obliged to the 
second referee for his significant comments. 



APPENDIX A: GIBBS INEQUALITY DERIVED FROM CONVEXITY OF IN- 
FORMATION CONTENT AND APPROPRIATE EXPECTATION VALUE 



As presented in (|8|), the Kullback-Leibler entropy is generally defined by means of the 
information content, 



K q ( P A \ T p) = E qjpA [h{pf)-k{pi 

Our result ( Jl6| ) implies that 

iAP2)-iApi)=p^ i9) iJf\ 



(Al) 



(A2) 



11 



Substituting this relation into (|A1|) , the Kullback-Leibler entropy is 



P 



13 



E A 
1:P 



If we take an unnormalized expectation value: 



Ef^ix] = Y.pf q)+1 * 



1=1 



then 



K q (P 



P 



(A3) 



(A4) 



(A5) 



I q (p) is a convex function with respect to p G [0, 1] for any q G R + . Thus, we can use 
Jensen's inequality |J: if / is a convex function and X is a physical quantity (random 
variable in mathematics), then 



E[f(X)]>f(E[X}) 

where E is the usual expectation value when q = 1. Therefore 

/ w 



kAp 



p 



Eptl 



'■n B \ ( W rr 8 ' 

Pi \ \ r / „APi 



i=l 



^Pt. 



1 v A 

8=1 Pi , 



If we take a normalized expectation value: 



p\ 



=i E Pj 

3=1 " 



\i=i 



-x- 



satisfies 



0. 



(A6) 



(A7) 



(A8) 



then the same Gibbs inequality as (|A7|) can be obtained. When ip (q) = q—1, the expectation 
values (|A4|) and ( |A8| ) coincide with q-expectation value defined by fl2"0|) and normalized q- 



expectation value defined by (|3Q|) , respectively. Thus, if an expectation value is chosen 
appropriately for a given self-information, then the nonnegativity condition of the Kullback- 
Leibler entropy is held. 



APPENDIX B: SYSTEMATIC FORMULATIONS OF VARIOUS NONEXTEN- 
SIVE ENTROPIES 

The proposed procedure for defining nonextensive entropy using an information content 
and an expectation value is applicable to the systematic formulations of various nonexten- 
sive entropies such as Kullback-Leibler entropy (relative entropy) and mutual entropy in 
accordance with the formulations in Shannon information theory 
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In nonextensive systems, an information content and two expectations are given in section 
III. Therefore we can formulate various nonextensive entropies in the following two cases. 

Case 1 : the information content ([16]) and the g-expectation value (|2 



Case 2 : the information content (|T6"D and the normalized ^-expectation value (|30|) 
In each formulation <p (g) = q — 1 is used as the most typical function of tp (q) . 



OF2* 

1. Various nonextensive entropies using g-expectation value E q [■ 



The information content (|T^) and the (/-expectation value (|20|) are applied to the defini- 
tions of various entropies as follows. 



(0-1) (nonextensive joint entropy) 
(0-2) (nonextensive conditional entropy) 



W A W b 



AB 

ij 



-EE (pf) 

i=l j=l V ' 

q-1 



q,p A 



S n 



i=i 



j=i V p * 



9-1 



E (pf) -EE fef) 

1=1 V ^ 8=1 7 = 1 V 7 



V AB 



q W A W b 

' - E 

for i = 1, 



, W A , 



(0-3) (nonextensive Kullback-Leibler entropy) 



P 



org 

q,p A 



i,(p? 



hi 



PI 



W u f v A\1 



1-9 



(0-4) (nonextensive mutual entropy) 



p A p B 



W A Htf , . ,,\ / !>■'■' 



^org 



q,p 

ab \ q 



Iq (Pfpf) - h (Pf 



1-q 



(Bl) 



(B2) 



(B3) 



(B4) 
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2. Various nonextensive entropies using normalized (/-expectation value E™ or [•] 



The information content (|T^) and the ^-expectation value (|30|) are applied to the defini- 
tions of various entropies as follows. 



(N-l) (nonextensive joint entropy) 



W A W b 



nor 

q,p AB 



Iq(p. 



AB 



-EE (p 

i=l 7 = 1 V 



w A w B f N 
(q ~ 1 E E (p^) 



(N-2) (nonextensive conditional entropy) 



W A 

E 



where 



V AB 



V AB 



S n 



P. 



AB 1 



Pi 



w B 

i-E 



(9-1) E 



/ A! 

EM 



J 5 



for z 



W A 



(N-3) (nonextensive Kullback-Leibler entropy) 



K or ( 



P 



P nor 



>9 (Pf ) " h (. 



Pi 



(N-4) (nonextensive mutual entropy) 



AB 



V 



w A w B 



P A P B 



yy A ii ti f . n \ / jy" 



9.P 
„AB \ 1 



(1 - q) E (pf ) 



I q (p?pf)-I g (pf) 



W A W b 



;i - g) e e (p 



s=l i=l 



(B5) 



(B6) 



(B7) 



(B8) 



Note that both of nonextensive mutual entropies respectively defined by (|B4j) and ( |B8 
are clearly symmetric with respect to "|pf | <-> {pf }" m each formulation. Furthermore, 
the nonnegativity of mutual entropies (|B4|) and ( P38|) is directly derived from that of the 
nonextensive Kullback-Leibler entropy, i.e., 



(p^; P B ) = #g (p^ ||pV*) > for any q G R + , 



(B9) 
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with equality pf^ = pfpf for any i — 1, • • • , Wa and j — 1, • • • , W 7 ^. 
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